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CURRENT STATUS, 
CHALLENGES, AND 
OPPORTUNITIES OF THE 
INTELLIGENT TUTORING 
SYSTEM (ITS) IN DEVELOPING 
COUNTRIES IN ASIA 


Kaushal Kumar Bhagat, Ma. Mercedes T: Rodrigo, and 
Chun-Yen Chang 


Introduction 


Currently, education systems in the Asian region are undergoing extraordinary developments: 
a growing number of people are gaining access to formal and informal learning, curricula are 
diversifying, and educational institutions are experimenting with new and innovative forms of 
delivery (Asian Development Bank, 2011). Despite this, Asian educational systems continue to 
confront four overarching challenges — namely, maintaining education quality, improving the 
relevance of curricula, improving on expenditure of financial resources, and balancing expan- 
sion with greater equity. One technological solution that has the potential to offer quality 
instruction to many students is the Intelligent Tutoring System (ITS). 

In recent years, an increasing number of researchers have shown interest in the development 
and/or deployment of ITSs for and in developing countries.! In particular, ITSs are seen to have 
the potential to augment or support overstretched educational systems (Nye, 2015), thereby 
becoming instruments of equity, quality, and efficiency. 

This chapter provides an overview of ITS-related research undertaken in Asian developing 
countries. The chapter begins with a brief overview of the development and nature of ITSs. It 
then outlines the methods for selecting and analyzing the literature used for the study, describ- 
ing the research foci of these studies and highlighting the findings in relation to the effectiveness 
of ITSs developed or deployed in these countries. The chapter concludes with a discussion of 
the limitations of current ITS research and identifies potential areas for further study. 


A brief overview of Intelligent Tutoring systems 


The evolution of ITSs follows developments in computer technology and cognitive psy- 
chology. According to Shute and Psotka (1994), the first time computerized technology was 
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used as a teaching aid was in 1926. This machine administered multiple-choice questions to 
students and provided immediate feedback on their responses. However, this technology was 
not underpinned by a learning theory; it simply presented a series of questions. The mid- 
1950s saw the rise of computers that had a greater capacity for logical decision-making and for 
data storage and manipulation. Around the same time, the educational psychology literature 
asserted that people tend to learn most effectively through individualized tutoring, and com- 
puters were considered to be the most appropriate vehicle to carry out this type of instruc- 
tion. Computer-based programmed instruction (PI) emerged in the 1960s in which students 
were taken through problem-solving activities in a lock-step fashion. This was followed by 
computer-assisted instruction (CAI) based on insights from behaviorist stimulus-response psy- 
chology. CAI differed from PI in that it had the capacity to branch out into specific content 
or remediation appropriate to the students’ progress. Intelligent CAI (ICAI) improved upon 
these branching capabilities by classifying and diagnosing student errors in order to be able to 
address misconceptions with greater specificity. Later on, the education and artificial intelli- 
gence disciplines began to collaborate to develop more efficient methods to store and retrieve 
knowledge, to reason using induction and deduction, and to communicate with students. 
These investigations gave rise to ITSs. 

An ITS is a computer-based learning tool that makes use of artificial intelligence to create 
educational environments that respond both to the learner’s level and needs and to the instruc- 
tional agenda (Graesser, Conley, & Olney, 2012). The ITS provides learners with personalized 
hints, guidance, remediation, and emotional support in order to achieve a learning goal. 

In Shute and Psotka’s (1994) analysis, ITSs must be able to diagnose students’ knowledge 
using principles rather than pre-programmed logic and to decide on what it should teach the 
student next. Thus, ITS architecture can be divided into four major components: a domain or 
expert model, a learner or student model, a tutor or pedagogical model, and a user interface. 
The domain model contains a representation of the content (skills, knowledge, procedures, and 
so on) that the student is required to learn. It also includes information pertaining to common 
student errors and misconceptions so that ITSs are able to provide appropriate assistance when 
such errors or misconceptions occur (Sottilare, Graesser, Hu, & Holden, 2013). 

The learner model contains the psychological states of learners (Sottilare et al., 2013). While 
in the 1970s and 1980s ITSs tended to track students’ knowledge and skills (Shute & Psotka, 
1994), over the last two decades there has been a growing interest in tracking non-cognitive 
states, such as student emotion and motivation (Damasio, 2000; Picard, 1997). 

The tutor model is responsible for regulating the interactions between the student and the 
system (Ahuja & Sille, 2013). Informed by both the domain model and the learner model, 
the tutor model selects teaching strategies that will most effectively assist the learner’s progress 
(Sottilare et al., 2013). In Cheng, Zhao, Xu, and Li (2014), with the Personalized Mathematics 
ITS, for example, each student begins by taking a diagnostic test which establishes his or her 
prior knowledge. The tutor model then suggests lesson preparation based on the results of the 
diagnostic. 

Finally, the user interface draws input from the learner and conveys this information to the 
different models (Sottilare et al., 2013). The interface also receives output from the tutor model 
and displays it to the learner in a form that the learner can understand. An interesting example 
of the user interface is Scooter the Tutor, an animated dog that is part of the Scatterplot Tutor 
described in Rodrigo et al. (2012). Scooter detects a form of non-learning behavior called 
“gaming the system”, which refers to student abuse or misuse of the tutoring system in order 
to progress through the material without actually learning and which usually manifests as sys- 
tematic guessing or repeated hit requests (Baker et al., 2006). When the student uses the tutor 
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properly, the user interface displays a happy version of Scooter, i.e., Scooter giving a thumbs-up 
or wagging his tail. However, if the student games the system, Scooter turns red and appears 
angry. 

The ability of these systems to adapt to learner needs has had a positive impact on their 
effectiveness as carriers of educational content. A recent meta-analysis by Ma et al. (2014) found 
that students using ITSs tended to out-perform those provided with teacher-led, large-group 
instruction; non-ITS computer-based instruction; and traditional learning materials, such as 
textbooks. The only educational delivery mechanism that matched ITSs’ effectiveness was one- 
on-one or small-group human tutoring. Further research findings relating to the effectiveness 
of ITSs are also described. 


Data collection and analysis 


In order to identify relevant studies on ITS for inclusion in the analysis for this chapter, we 
conducted searches of various citation databases (Scopus, Web of Science, ERIC, Google 
Scholar, and PsycINFO) using a combination of keywords, such as intelligent tutoring systems, 
achievement, effectiveness, outcome, and learning. We also conducted Internet searches of conference 
proceedings. The initial screening yielded 40 abstracts published between 2010 and 2015 that 
were related to ITS based on their research foci. The research foci fell into two groups. The 
first group comprised theoretical papers which focused on the ways in which ITSs should be 
designed, work, or behave, while the second group included studies relating to ITS develop- 
ment in terms of actual programming. The task of screening all the abstracts generated by the 
search in order to identify their relevance to the study was completed independently by two of 
the authors of this chapter. The overall inter-reliability was 0.75, which is considered to be an 
acceptable result. 


Geography and foci of ITS research in Asia 


The papers surveyed originated from China, Hong Kong, Indonesia, Malaysia, Pakistan, Pal- 
estine, Philippines, Taiwan, Turkey, and Thailand. As highlighted earlier, the research foci 
ranged from theoretical contributions to ITS implementations and deployments. Table 36.1 
summarizes the studies that were examined. 


Table 36.1 Included studies and their research foci 


Authors Country Subject area Research foci 
Cheng et al., 2014 China Mathematics Development 
Chung, Nagai, & Rodrigo, 2013 Philippines Japanese Development 
Hwang, Tseng, & Hwang, 2008 Taiwan Mathematics Development 
Kazi, Haddawy, & Suebnukarn, 2010 Thailand and Pakistan Medicine Development 

Li et al., 2010 China Science Development 
Phobun & Vicheanpanya, 2010 Thailand Not applicable Theory-oriented 
Regalado et al., 2015 Philippines Filipino Development 
Rodrigo et al., 2012 Philippines with US Mathematics Theory-oriented 

collaborator 

San Pedro, Baker, & Rodrigo, 2014 Philippines and US Mathematics Theory-oriented 
Sun, Li, & Xia, 2013 China Not applicable Theory-oriented 
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THEORY-ORIENTED ITS STUDIES 


The papers we classified as “theory-oriented” focus on the foundational concepts that can be 
used as bases for designing or building an ITS. As such, they either attempt to explain ITS 
architecture or propose augmentations or improvements but do not address issues of implemen- 
tation. Some of these studies make use of existing systems. However, rather than discussing the 
processes by which these ITSs were built, they use these systems as data-gathering instruments. 
The analysis of the data reveals insights that are then used to inform learner, pedagogical, or 
domain model design or user interface implementation. 

By way of example, Phobun and Vicheanpanya (2010) describe the architecture and flow 
of an adaptive intelligent tutoring system (AITS). The authors combine the features of an ITS 
with those of adaptive hypermedia. Adaptive hypermedia shares some of the characteristics of 
ITSs insofar as content and navigation options are adapted based on learner goals and knowl- 
edge. The authors distinguish between the two types of systems based on applicability, where 
adaptive hypermedia seems best suited for teaching declarative knowledge, while ITS is more 
appropriate for problem-solving. The AITS proposed by Phobun and Vicheanpanya includes 
a system that walks the learner through a problem-solving process while he or she is learning 
materials usually found in ancillary documents, such as textbooks, and takes the form of adap- 
tive hypermedia. One interesting detail about the architecture of the AITS is that the authors 
recommend the expert model, rather than the teaching or pedagogical model, to compare 
the domain knowledge against the knowledge of the learner. Having identified the learner’s 
knowledge deficits, the expert model presents the learner with a list of recommended content. 

Some theoretical papers describe how one or more of the models within the ITS architecture 
can be expanded to include new capabilities and features. The work of Sun et al. (2013) 
attempts to address ITSs’ lack of emotion and how this shortcoming might undermine learners’ 
motivation and learning. They describe how emotion recognition can be built into these 
systems. They begin by taking a picture of the student while the student’s face has a neutral 
expression. A camera trained on the student’s face then feeds facial expression data to the ITS. 
The ITS compares the facial expression data with the neutral expression and analyzes whether 
the student is expressing a positive or negative affective state. If the ITS determines that the 
student is distressed, it can search for hints that fit the problem that the student is currently 
trying to solve. The expert model of the ITS can then tag lessons with an emotional difficulty 
rating, depending on how students fare. 

Several studies conducted in Asia made use of existing ITSs as a test bed for experiments, 
rather than as a regular part of the school curriculum. Researchers responsible for these experi- 
ments were usually investigating student-related phenomena, such as learning gains, prefer- 
ences, behaviors, or affective states. Studies of this type contribute to ITS research in that they 
lead to more sensitive and nuanced student models. 

A study by Rodrigo et al. (2012) used an ITS developed in the United States to investigate 
the effects of Scooter the Tutor, an embodied conversational agent (ECA), on the behavior and 
affective states of students in the Philippines. In prior studies conducted in the United States, 
Scooter was shown to discourage student off-task behaviors. However, this finding was not 
replicated in the Philippine study. Indeed, while US students disliked the ECA, students in the 
Philippines thought of the ECA as useful, helpful, and caring. The findings from the Rodrigo 
et al. study suggest that perceptions of the same ECA are culturally influenced and that design 
decisions should be informed by ethnographic studies of target populations. 

Using the same data set, San Pedro et al. (2014) studied the relationship between affect 
and student carelessness, which is defined as student failure to provide a correct answer on a 
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question whose skill the student has already mastered. They found that carelessness was com- 
mon among students who were highly engaged and less common among students who were 
confused and bored. This did not indicate that engaged students committed fewer errors; rather, 
it meant that their errors, when committed, were likely to be careless, most probably attribut- 
able to overconfidence. Students who were bored or confused, in contrast, tended to make 
honest errors, i.e., errors that stemmed from a genuine lack of knowledge. 


ITS development 


Much of the ITS literature from Asia focuses on the development of ITSs across various 
domains. Papers often discuss the architecture of the ITS, emphasizing reinterpretations or 
departures from the classic modules. 

Salinlahi (Regalado et al., 2015) is an ITS that teaches Filipino grammar and sentence con- 
struction. It offers students two types of exercises: translation and creation. Salinlahi’s develop- 
ers describe its architecture in terms of the user-interface, expert, tutoring, and student models, 
which were highlighted earlier. The user interface is responsible for interactions between the 
student and the system. The expert model contains two analyzers, one for each type of exercise: 
the translation analyzer, which cross-checks the student’s response against the expected answer, 
and the creation analyzer, which parses student input and determines its distance from a pre- 
ferred response. The student model represents that which the student has and has not mastered. 
Finally, the tutoring model tracks student attempts, collaborates with the expert model regard- 
ing student assessment, and updates the student model. 

Chung et al.’s (2013) ITS also focuses on language but with a more limited scope, namely 
the teaching of Japanese language particles. The paper discusses the student model as an overlay 
of the expert model, but does not provide information on how the domain knowledge is struc- 
tured. Chung et al.’s paper places emphasis on the design of the user interfaces, rationalizing 
design choices using the cognitive theory of multimedia learning (Mayer, 2005). The princi- 
ples that the authors employ include the focus on task-relevant information, the limitation of 
unnecessary information, and the guiding of student attention. 

The systems of Regalado et al. (2015) and Chung et al. (2013) tend to follow classic ITS 
architecture. Other researchers innovate with their systems’ underlying structures. 

For instance, Kazi et al. (2010) describe Medical Tutor Employing Ontology for Robust- 
ness (METEOR), an ITS for problem-based learning in medicine. METEOR is built on 
top of the Unified Medical Language System (UMLS), a knowledge base containing over 
2 million medical concepts. These concepts and their causal relationships are codified based 
on human domain experts. A solution to a problem-based scenario usually adds causal links 
to the knowledge base and requires about 3 to 4 hours to code. Instead of building a separate 
student model, the authors propose using the expert knowledge base to assess directly student 
solutions for partial correctness and to generate hints. Evaluations of the resulting system show 
that METEOR’s hints were sometimes better than those of typical problem-based learning 
tutors. Furthermore, the approach of bypassing the student model saved time and effort when 
expanding the ITS. 

Li et al. (2010) developed a science workbench that enabled students to conduct virtual 
experiments regarding light and reflection. In the periscope experiment, for instance, students 
must position mirrors so that the images in view of one end of the periscope become visible 
to the eye at the other end. The learning environment is a network-based system of multiple 
agents that share resources and coordinate to achieve common ends. Following the client- 
server architecture, students interact with a teacher and student agent on the client side. The 
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teacher agent is responsible for the tracking and analysis of student characters as well as the 
selection of teaching strategies. The student agent manages interactions between the student 
experiment system and the rest of the ITS. A database server houses data regarding student 
character, experiment teaching strategies, experiment processes, teaching resources, and labora- 
tory apparatuses. A collaboration server manages communications among the various agents. 

The study by Hwang et al. (2008) contains an extensive discussion of how questions map 
to specific mathematical concepts, such as coefficients, exponents, and polynomial arithmetic. 
The representation also contains information regarding the interdependencies among concepts. 
When evaluating student performance, the software is able to identify which lessons the system 
needs to provide to the student and in what sequence. An example of this is if a student fails to 
meet the mastery criteria of both the exponent and distributive law concepts, the system must 
then enhance the student’s “exponent->multiplication of polynomial” learning path and the 
“distributive law->multiplication of polynomial” learning path. 

Finally, Cheng et al. (2014) discuss a mathematics ITS (MITS) that they developed in 
response to the need for mathematical content in Chinese. The MITS provides learners with 
personalized support in the form of online courses with mathematics-related web pages and 
videos. MITS is also able to respond to student inquiries and clarifications with extensive tutor- 
ing information. MITS is organized as a client-server system with three layers: expression, func- 
tion, and data. The client is the expression layer — the equivalent of the user interface model 
in classic ITS architecture; the function layer refers to the machine intelligence that provides 
the rationale for the ITS’s next actions; and the database layer contains course content and user 
data. As with the system of Hwang et al. (2008), the mathematics content of MITS is organ- 
ized as both units of concepts and the relationships among them. The learner model tracks the 
student’s mastery of each unit and then recommends the next lessons accordingly. 


Learning effectiveness of ITS: a meta-analysis of selected 
Asian research 


When ITSs are deployed to influence student achievement, results tend to compare favorably 
against other teaching tools and methods. To the best of our knowledge, only two meta-analysis 
studies about the learning effectiveness of ITS have been published thus far. Steenbergen-Hu 
and Cooper (2013) conducted a meta-analysis on the effectiveness of ITS in mathematics learn- 
ing on K-12 students. They found that, overall, there was a very small positive effect on K-12 
students’ mathematical learning compared to the traditional mode of teaching. Ma, Adesope, 
Nesbit, & Liu (2014) conducted a meta-analysis to compare the learning outcomes of the 
learners between ITS and non-ITS learning modes. They found a positive effect of ITS-based 
learning, with an effect size (g = +0.41). Although these two studies have produced empirical 
evidence of the effectiveness of ITS, the samples also contain data from Western countries. 
Therefore, in this section, we present the results of our meta-analysis on the learning effective- 
ness of ITS in Asia specifically, based on a selection of relevant studies identified during the 
literature search. 


Inclusion criteria 


The papers included in the meta-analysis were selected on the basis of a number of criteria, 
namely that they reported on original data; that samples were taken from Asia; that they com- 
pared learning outcomes from ITS learning environments with those from a non-ITS learning 
environment; and that sufficient quantitative data was presented in order to calculate effect sizes 
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(articles in which the sample size for each group was not cited were excluded). Based on these 
criteria, a total of 12 studies qualified for our meta-analysis (see Appendix 1). While our sample 
size might seem limited, Pigott (2012) contends that a minimum of two studies is sufficient to 
conduct a meta-analysis of this kind. 


Coding of study features 


In order to identify the substantive features that might contribute to the variation, the selected 
studies needed to be coded. The substantive features were coded according to the following 
categories: 


e Grade level: Elementary School (1-6), Junior High School (6—9), High School (10-12), or 
Undergraduate 

e Types of Asian country: Developed or Developing 

¢ Subject domains: Mathematics, Earth Science, Computer Programming, or Others 


Calculation and analysis of effect sizes 


Borenstein, Hedges, Higgins, and Rothstein (2009) recommend the following steps to conduct 
a meta-analysis: (1) calculate the effect sizes for each article, (2) calculate the weighted mean 
effect size across articles, (3) determine the confidence interval for the average effect size, and 
(4) undertake a heterogeneity test analysis (QB). 

After calculating the individual effect size for all 12 studies, comprehensive meta-analysis 
software was employed to calculate the unbiased effect size (Hedge’s g), as well as the values for 
the test of heterogeneity (QB). In the present study, we calculated only one effect size for each 
article. In those cases where the studies had multiple effect sizes, these were integrated into one 
effect size (following Borenstein et al., 2009). Appendix 1 shows the coded information and 
effect sizes of the selected 12 studies. 


Results 


OVERALL EFFECTS 


As indicated in Table 36.2, the overall weighted effect size is +0.570, with a 95% confidence 
interval of 0.114-1.026. Q statistics show that the effect sizes in the meta-analysis were hetero- 
geneous (Q = 113.6, df= 11, p < 0.00), which indicates that there are differences among the 


Table 36.2 Overall effect sizes 


k ES SE Variance 95% confidence Test of mean Test of heterogeneity in 
interval effect sizes 
Lower Upper Z P Q-value dQ) p 


Fixed 12 0.229 0.066 0.004 0.1 0.359 3.474 0.001 113.6 11 0.000 
Random 12 0.570 0.233 0.054 0.114 1.026 2.447 0.014 


Effect sizes of learning achievement for moderator variables 
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effect sizes resulting from factors other than simple sampling error. Therefore, a random effects 
model was reported (Borenstein et al., 2009). In order to explain this variance, substantive 
features (e.g., type of Asian country, grade level, subject domain) were used to model some of 
the variation. 


GRADE LEVEL 


Table 36.2 indicates that junior high school students had a high effect size on learning achieve- 
ment (g = 1.684, z = 5.996, p < .05), while elementary school students (g = —0.217, z= —1.968, 
p < .05) had a small effect size. High school (g = 0.129, z = 0.63, p > .05) and undergradu- 
ate (g = 1.031, z = 1.598, p>.05) students did not show any significant effect sizes. The QB 
achieved significance (QB = 52.10, p<.05), indicating that the mean effect size differs signifi- 
cantly between the categories. 


TYPES OF ASIAN COUNTRY 


Table 36.2 indicates that developing countries in Asia had a high effect size on learning achieve- 
ment (g = 1.223, z = 3.685, p < .05), while developed countries did not show any significant 
effect size (g = —0.099, z = —0.507, p > .05). The QB achieved significance (QB = 11.77, 
p < .05), indicating that the mean effect size differs significantly between the categories. 


SUBJECT DOMAINS 


Table 36.3 indicates that other subjects had a high effect size on learning achievement 
(e = 1.562, z = 2.599, p < .05), while mathematics (g = 0.561, z = 1.515, p > .05), 


Table 36.3 The learning-achievement effect sizes of categories and their related moderator variables 


Category k G Z 95% CI QB 
Grade level 52.10* 
1. Elementary school 1 —0.217 —1.968* —0.434,—0.001] 

2. Junior high school 1 1.684 5.996* [1.133,2.234] 

3. High school 5 0.129 0.63 [-0.273,0.532| 

4. Undergraduate 4 1.031 1.598 [-0.230,2.255] 

Types of Asian country 11.77 
1. Developed country 6 —0.099 —0.507 [-0.483,0.284] 

2. Developing country 6 1.223 3.685* 0.572,1.873] 

Subject domains 6.725" 
1. Mathematics 4 0.561 1515 [-0.615,1.287] 

2. Computer programming 4 0.401 0.860 [-0.513,1.314] 

3. Earth science 2, —0.204 —0.566 —0.912,0.503] 

4. Others 2 1.562 2.599% [0.384,2.740] 


*p < .05 


Note: Regalado et al. (2015) do not mention grade level. Therefore, for grade level, the total number of 
studies is 11. 
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computer programming (g = —0.401, z = —0.860, p > .05), and earth science (g = —0.204, 
z = —0.566, p > .05) did not show any significant effect size. The QB achieved significance 
(QB = 7.76, p < .05), indicating that the mean effect size differs significantly between the 
categories. 

Analysis of empirical research from Asia on the learning effectiveness of ITS has revealed 
that the overall effect of ITS is better than both the traditional mode of learning or not 
using ITS as an intervention, with a moderate effect size of 0.570. Through the analysis 
of moderator variables, we found that ITS has been applied to various grade levels, types 
of Asian country, and subject domains. The effect of such usage was greater for develop- 
ing countries than for developed countries in Asia, and the usage for junior high school 
students was more effective than for elementary, high school, and undergraduate students. 
The novelty effect may be the reason for better performance of ITS in developing countries 
than in developed countries. Chien, Chang, and Chang (2016) reported that new technol- 
ogy usually receives greater attention from students when it is incorporated into the class- 
room. As a result, there are positive gains in learning achievement owing to an increase in 
attention. 


Limitations of ITS research in Asia 


Several observations can be made based on our review of the literature. On a positive note, 
within Asia, there is a research community that is interested in contributing to the theoretical 
foundations, construction, and deployment of ITSs. When deployed, ITSs are effective carriers 
of educational content. Tools such as these are needed in the Asian region in general and in 
developing countries in particular. 

However, Asian ITS research exhibits a number of limitations, such as the following: 


e The research tends to be compartmentalized, i.e., studies tend to focus on only one 
aspect — ITS theory, development, or implementation — rather than on the entire lifecycle 
from conceptualization to classroom use. As such, the Asian region seems to lack end-to- 
end solutions where theory, implementation, deployment, and evaluation form a virtuous 
cycle. 

e Many contributions to theory do not address implementation. 

e  ITSs that are developed locally are tested in laboratories rather than schools. 

e — ITSs that are used in schools tend to be developed in other countries and only for a limited 
time as part of formal experiments. 

e Finally, our review did not yield any evidence of the integration of ITSs as part of the 
regular school curriculum. 


What are the possible reasons for these limitations to the research? To begin with, there are 
instances in which ITSs require Internet connectivity and relatively high computer hardware 
specifications. However, despite the growing availability of computers and the Internet, many 
schools in developing countries still lack infrastructure — electricity, telecommunications, and 
computer hardware (Nye, 2015) — to support these applications. Some Asian countries have 
outperformed international standards of technology-assisted instruction. Nevertheless, the lack 
of infrastructure in other countries prevents them from benefiting from these resources. In- 
country differences are also stark: while urban areas adopt technology quickly, remote areas do 
not (UNESCO Institute of Statistics, 2014). 
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Related to the lack of infrastructure is the paucity of culturally appropriate content (Nye, 
2015). The examples in this paper notwithstanding, ITSs tend to be built in Western, indus- 
trialized, educated, rich, and developed countries (Blanchard, 2012). Hence, the language and 
culture (symbols, contexts, situations, and motivations) of the ITSs may not be a good fit for 
the target country’s students. 

Another major impediment is students’ lack of technology skills. In schools with few comput- 
ing facilities, students receive limited opportunities to work with the machines on a one-to-one 
basis. Furthermore, they almost never use learning software, such as ITSs (Rodrigo et al., 2014). 
When new computer-based educational interventions are introduced, students typically need 
support — at first to become comfortable with the technology and then to start benefiting from it. 


Opportunities for further research 


The results of our study have several implications for educators and policy-makers in develop- 
ing countries. The gaps and barriers open many opportunities both for the generation of new 
knowledge and for greater societal impact. Within the Asian region, in particular, continu- 
ing research, development, deployment, and evaluation of ITSs has the potential to promote 
greater educational quality, equity, and efficiency. 

For example, there is a need to develop ITSs beyond universal subject areas such as math- 
ematics and science. Subject areas that have a more localized interest (e.g., local languages and 
local histories) are ripe for technology interventions. ITSs can be used to preserve and promote 
local culture — both tangible (art, crafts, and historical sites) and intangible (social values, cus- 
toms, and religious beliefs) (Mortara et al., 2014). 

ITSs could also be designed for deployment on mobile phones and tablets — technologies 
that are far more ubiquitous than personal computers. However, as highlighted earlier, although 
Internet accessibility is increasing, many parts of Asia still lack electrical and telecommunications 
infrastructure (Nye, 2015; UNESCO Institute of Statistics, 2014). It would therefore be ben- 
eficial if ITSs were designed for stand-alone use. If deployed on handheld devices, ITSs should 
attempt to maximize battery life by being graphics-light and by minimizing the use of device 
components such as the camera, GPS, and accelerometer. 

ITSs that already exist, and which have been proven to be effective in laboratory studies or 
in other countries, could be deployed on a wider scale. This may require translation to the local 
language as well as the localization of content. Imagery and symbolisms (e.g., fruits, vegetables, 
animals, and sports) would need to be relevant to the target audience. 

The effects of deployed ITSs could be monitored and studied. Aside from tracking learning, 
researchers should be particularly mindful of ITSs with affective interventions, as students from 
different cultures respond to these interventions in different ways. The findings of Rodrigo 
et al. (2012), for example, showed that Philippine students regarded Scooter the Tutor very 
differently from their US counterparts. Culture influences how people regard authority, how 
individualistic or collective their thinking is, the degree to which they have to maintain smooth 
interpersonal relationships, and so on. These variations interact with affective interventions to 
produce different effects. 

Finally, investigations of ITS deployments present opportunities for collaboration between 
researchers from different countries. Collaborations can lead to the transfer of technology from 
more- to less-developed countries, which, in turn, can lead to more universally applicable ITSs 
and increase access to quality instruction. International collaborations can also contribute to the 
development of ITSs that are more culturally sensitive. 
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Appendix 1 


CODED INFORMATION OF 


SELECTED STUDIES POR 


META-ANALYSIS 


Study Sample size Effect size Grade level Subject domains Type of Asian 
country 
Abu-Naser (2009) 62 0.686 Undergraduate Computer Developing 
Programming 

Bulut Ozek, 108 1.907 Undergraduate Computer Developing 
Akpolat, and Programming 
Orhan (2013) 

Chen (2008) 220 —0.217 Elementary Mathematics Developed 

Chen (2011) 145 —0.616 Undergraduate Computer Developed 

Programming 

Chien, Yunus, 62 1.684 Junior high Mathematics Developing 
Suraya, Ali, and school 
Bakar (2008) 

Huang, Yeh, Li, 72 —0.003 High school Earth Science Developed 
and Chang 
(2010) 

Hwang, Tseng, and 76 0.746 High school Mathematics Developed 
Hwang (2008) 

Keun-Woo, 115 —0.222 High school Computer Developed 
EunKyoung, Programming 
and YoungJun 
(2010) 

Regalado et al. 27 1.011 Not available Others Developing 
(2015) 

Rodrigo et al. 126 0.159 High school Mathematics Developing 
(2012) 

Wang, Rosé, and —0.264 High school Earth Science Developed 
Chang (2011) 

Yang, Leung, Yue, 52 2.217 Undergraduate Others Developing 


and Deng (2013) 
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Note 
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